• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Provided is a technology which facilitates a search for an object such as a falling object. This object search system 1 is provided with: a detection device 110 which detects position information on a falling object 106; and a scope 109 which is worn by a user and provided with a display unit that displays an augmented reality space in a real space. The detection device 110 detects an object by using a radar device 100. A radar control device 101 calculates the detected position information on the object and transmits the calculated position information to the scope 109 through a wireless transmission device 103. The scope 109 displays the position information, which pertains to the falling object 106 and is acquired from the detection device 110, on an augmented reality space with a specific indication (an arrow or a wave shape, etc.).
    公开号:EP3690626A1
    申请号:EP18860092.8
    申请日:2018-08-28
    公开日:2020-08-05
    发明作者:Yosuke Sato；Kenichi Kashima
    申请人:Hitachi Kokusai Electric Inc；
    IPC主号:G06K9-00
    专利说明:
    [0001] The present invention relates to an object search system, an object search device and an object search method, and particularly to an object search system, an object search device and an object search method which display an augmented reality space, made out of a synthesized image, on a real space. BACKGROUND ART
    [0002] There is a system which detects an obstacle, a fallen object or the like (hereinafter referred to as a fallen object) from a vast area (a work site, a ground, an inside of a facility, a parking area, a road, a railway, a runway or the like) using a radar or the like. Such a system is capable of identifying a position (coordinates or the like) of the fallen object with some precision based on a result of the detection by the radar.
    [0003] Once a fallen object is detected by the radar, a worker or the like of a management company goes to collect the fallen object. A conventional practice is that the worker or the like searches for the fallen object while seeing a map based on the coordinate information about the fallen object. For example, the map is displayed on a tablet, and a position of the fallen object is highlighted on the map.
    [0004] Meanwhile, techniques using the AR (augmented reality) technique have been proposed in recent years (see, for example, Patent Literature 1 and Patent Literature 2). Patent Literature 1, for example, discloses a technique of displaying various types of additional information (AR information) on a head-mounted display device by superposing the various types of information on a real space (external scene). On the other hand, Patent Literature 2 discloses a technique of finding a target material or part from a large number of articles placed in a storage yard. CITATION LISTPatent Literature
    [0005] Patent Literature 1: Japanese Patent Application Publication No. 2015-075832 Patent Literature 2: Japanese Patent Application Publication No. 2011-242591SUMMARY OF INVENTIONTechnical Problem
    [0006] However, the search using the tablet involves a problem that the fallen object is difficult to find in the vast area with no mark. In addition, it is dangerous for the worker to perform the search while seeing the tablet. Furthermore, search time is not limited to daytime, and may be after dusk when there is a lack of sunlight. The search using the tablet involves a problem that a small fallen object is more difficult to search for in a situation where it is dim. The technique disclosed in Patent Literature 1 or 2 is not designed to search a vast area for an object. There has been a demand for a new technique.
    [0007] The present invention has been made with the above situation taken into consideration, and an object of the present invention is to solve the above problems. Solution to Problem
    [0008] The present invention is an object search system including: a detection device which detects position information about an object; and an object search device to be worn by a user, and including a display which shows an augmented reality space on a real space. The detection device includes a transmitter which transmits the position information about the object to the object search device. The object search device includes: a receiver which receives the position information about the object transmitted from the detection device; an acquirer which acquires position information about the object search device; and a display controller which shows a specific display to specify a position of the object on the display based on the position information about the object and the position information about the object search device.
    [0009] In addition, the display controller may show, as the specific display, a straight line indicating the position of the object such that the straight line extends to the position of the object.
    [0010] Furthermore, the display controller may show, as the specific display, a display indicating nearness to the object in the position of the object.
    [0011] Moreover, the display controller may increase the number of specific displays indicating the nearness, or make the specific display become larger, as the nearness to the object increases.
    [0012] Besides, the display controller may show, as the specific display, an arrow indicating a direction toward the object.
    [0013] In addition, the detection device may include a camera which captures an image of the object; the object search device may receive the image of the object; and the display controller may show the image on the display.
    [0014] The present invention is an object search device to be worn by a user, and including a display which shows an augmented reality space on a real space. The object search device includes: a receiver which acquires position information about an object to be searched for; an acquirer which acquires position information about the object search device; and a display controller which shows a specific display to specify a position of the object on the display based on the position information about the object and the position information about the object search device.
    [0015] In addition, the display controller may show, as the specific display, a straight line indicating the position of the object such that the straight line extends to the position of the object.
    [0016] Furthermore, the display controller may show, as the specific display, a display indicating nearness to the object in the position of the object.
    [0017] Moreover, the display controller may increase the number of displays indicating the nearness to the object, or make the display indicating the nearness become larger, as the nearness increases.
    [0018] Besides, the display controller may show, as the specific display, an arrow indicating a direction toward the object.
    [0019] An object search method according to the present invention includes: an object position acquiring step of acquiring position information about an object; a display position acquiring step of acquiring position information about an object search device to be worn by a user, and including a display which shows an augmented reality space on a real space; and a specific display step of showing, as a specific display reflecting a relationship between the position information about the object and the position information about the object search device, the position information about the object in the augmented reality space based on the position information about the object and the position information about the object search device. Advantageous Effects of Invention
    [0020] The present invention makes it possible to easily search for a fallen object in a situation where there is no information about a specific shape of the object. BRIEF DESCRIPTION OF DRAWINGS
    [0021] FIG. 1 is a block diagram illustrating a configuration of an object search system according to an embodiment. FIG. 2 is a block diagram illustrating a configuration of a scope according to the embodiment. FIG. 3 is a diagram illustrating examples of a mode in which a straight line and a ripple for indicating a location of a fallen object are displayed in the embodiment. FIG. 4 is a block diagram illustrating a configuration of the object search system according to the embodiment. DESCRIPTION OF EMBODIMENTS
    [0022] Referring to the drawing, detailed descriptions will be hereinbelow provided for an embodiment of the present invention.
    [0023] The embodiment will describe a technique of detecting an object in a vast area using the AR technique. Descriptions will be provided for an application example in which plant premises are searched for a fallen object by showing a specific display on a scope (object search device) including a transparent display.
    [0024] FIG. 1 is a block diagram illustrating a schematic configuration of an object search system 1 according to the embodiment. FIG. 2 is a diagram illustrating a scope 109. FIG. 2(a) is a block diagram illustrating an example of a display to be shown on the scope 109. FIG. 2(b) is a diagram illustrating a schematic configuration of the scope 109.
    [0025] As illustrated in FIG. 1, the object search system 1 includes the scope 109 to be used by a worker 108, and a detection device 110. Information about a fallen object 106 which is detected by the detection device 110 is shown on the scope 109.
    [0026] The detection device 110 includes: a radar device 100; a camera device 105; and a control/monitoring device 120 which controls the radar device 100 and the camera device 105. A surface condition of a road surface 107 is monitored using the radar device 100 and the camera device 105 to be installed within the premises. Where or how many radar devices 100 and camera devices 105 are to be installed is not particularly limited. Each radar device 100 and each camera device 105 are connected to the detection device 110 using optical fibers and the like, and can be remotely controlled.
    [0027] The radar device 100, for example, detects the existence of the fallen object 106 by: transmitting a predetermined electromagnetic wave (for example, in a millimeter waveband with a frequency of 60 GHz or more); and receiving a wave reflected off the fallen object 106. Based on a detection result, the camera device 105 takes an image of the fallen object 106.
    [0028] The control/monitoring device 120 includes a radar control device 101, a detection result display device 102, a wireless transmission device 103, and a camera control device 104.
    [0029] The radar control device 101 controls the radar device 100. Furthermore, the radar control device 101 has a function of performing signal processing on a signal reception processing result, and acquires fallen object information (specifically, position information (coordinates) and shape information) about the fallen object 106 by searching a predetermined range by the radar periodically or based on manipulation by an administrator or the like.
    [0030] Based on a result of the detection of the fallen object 106 by the radar device 100 and the radar control device 101, a camera control device 104 takes an image of the detected fallen object 106.
    [0031] The detection result display device 102 displays the signal reception processing result obtained through the signal processing by the radar control device 101. Specifically, the detection result display device 102 displays things such as: a map showing the position coordinates and fall position which are representative of where the fallen object 106 is; and a real image of the fallen object 106 which is taken by the camera device 105.
    [0032] Moreover, a wireless transmission device 103 is connected to the detection result display device 102, and transmits the fallen object information to portable terminals such as the scope 109 and a tablet (not illustrated) which the worker 108 carries.
    [0033] The transmission of the fallen object information to the scope 109, the tablet and the like may be preformed manually by the administrator who visually checks the existence of the fallen object 106 by looking at the detection result display device 102, or may be performed automatically based on the information detected by the radar control device 101.
    [0034] As illustrated in FIG. 2(b), the scope 109 includes a scope controller 151, a wireless transmitter 152, a display 153, a storage 154, and a position sensor 155.
    [0035] The position sensor 155 includes, for example, a GNSS sensor 156 and a gyro sensor 157. The GNSS sensor 156 detects a position of the scope 109, that is to say, a position of the worker 108. The gyro sensor 157 detects a direction in which the scope 109 faces.
    [0036] The wireless transmitter 152 acquires the fallen object information about the fallen object 106 and map information by making communications with the wireless transmission device 103 of the detection device 110, and stores the fallen object information and the map information onto the storage 154. Incidentally, the map information may be retained on the storage 154 in advance.
    [0037] The scope controller 151 controls the entirety of the scope 109. In this respect, referring to FIG. 2(a), descriptions will be provided for a display mode in which the scope controller 151 controls a display on the display 153. The scope controller 151 compares a direction in which the worker 108 looks and the position of the fallen object based on the position information (coordinates) about the scope 109 acquired by the GNSS sensor 156 as well as the gyro sensor 157, and displays a direction toward the fallen object 106 (that is to say, a direction in which the worker 108 is to go) on the display 153 using an arrow A. A distance to the fallen object may be indicated by changing boldness and length of the arrow A.
    [0038] In addition, the scope control 151 displays a line B (which may be a straight line or a broken line) extending from upper to lower sides on the field of view like a rainfall, in the position of the fallen object 106 which is acquired from the detection device 110. This display may blink. Furthermore, the distance between the fallen object 106 and the scope 109 may be displayed using a ripple C.
    [0039] FIG. 3 illustrates an example of a mode in which the straight line B and the ripple C indicating the location of the fallen object 106 is displayed. A display example on a more leftward position in FIG. 3 means being farther from the fallen object 106, while a display example on a more rightward position in FIG. 3 means being nearer to the fallen object 106. As the worker 108 wearing the scope 109 goes nearer to the fallen object 106, the display is performed using an increased number of more ripples C in order for the worker 108 to visually realize that the worker 108 goes nearer to the fallen object 106. In FIG. 3, a farthest fallen object is displayed using a single ripple C, while a nearest fallen object is displayed using three ripples C.
    [0040] It should be noted that the image of the fallen object 106 taken by the camera device 105 may be shown on the display 153 simultaneously. In this case, the image is displayed in an area where the image does not hinder the display of the fallen object 106 and the straight line B.
    [0041] Timings at which to search for the fallen object 106 are not limited to daytime. The timings may be after dusk when there is a lack of sunshine. Even in a situation where it is dim outside, the scope 109 displays things such as: the arrow A indicating the direction toward the fallen object 106; the line B extending from the upper to lower sides on the field of view like a rainfall which is displayed in the position of the fallen object 106; and the ripple C indicating the distance to the fallen object. The position of the fallen object, therefore, can be searched for by use of these displays.
    [0042] Referring to a flowchart in FIG. 4, integrated descriptions will be provided for the above series of the operations.
    [0043] Once the detection device 110 detects the fallen object 106 by the search of the radar device 100 (S10), the radar control device 101 acquires the position information about the fallen object 106 by identifying the position information from the detection information (S12). The radar control device 101 makes the camera device 105 take an image of the detected fallen object 106 by controlling the camera control device 104, and causes the shape of the fallen object 106 to be checked (S14).
    [0044] The detection result display device 102 displays the fallen object 106 which is detected by the radar device 100 and whose image is taken by the camera device 105 (S16), and transmits the object information about the fallen object 106 to the scope 109 (S18). The scope 109 performs the AR process on the object information about the fallen object 106 which is acquired from the detection device 110 and causes the display 153 to display the result (S20).
    [0045] Thus, the worker 108 can realizes specific information about the size, color and the like of the actual fallen object 106. Accordingly, it is possible to achieve an easy search, and a reduction in the search time.
    [0046] FIG. 2(a) and the like show an example of how the large fallen object 106 is detected. In a case where a vast area is searched for the fallen object 106 with a length of several centimeters, the use of the scope 109 according to the present invention makes it possible to reduce the labor and time to be required for the search to a large extent while securing the safety of the worker (a person engaging in the search).
    [0047] The present invention has been described based on the embodiment. The embodiment has been shown as an example. It should be understood by those skilled in the art that: various modifications can be made to the combination of the components; and such modifications fall within the scope of the present invention. For example, although in the above-discussed embodiment, the scope 109 includes the transparent type (optical transmission type) of display 153, the scope 109 may include a video transmission of display. In this case, the scope 109 is provided with an image capturing device which acquires an image of the outside.
    [0048] In addition, the above-discussed embodiment has provided the case where the scope 109 acquires the fallen object information (position information and the like) about the fallen object 106 by making the wireless communications with the detection device 110. Nevertheless, the method in which the object search device according to the present invention acquires the position information about the object from the outside is not limited. Reference Signs List
    [0049] 1object search system100radar device101radar control device102detection result display device103wireless transmission device104camera control device105camera device106fallen object107road surface108worker109scope110detection device120control/monitoring device151scope controller152wireless transmitter153display154storage155position sensor156GNSS sensor157gyro sensorAarrowBstraight lineCripple
    权利要求:
    Claims (12)
    [0001] An object search system comprising:
    a detection device which detects position information about an object; and
    an object search device to be worn by a user, and including a display which shows an augmented reality space on a real space, wherein
    the detection device includes a transmitter which transmits the position information about the object to the object search device, and
    the object search device includes
    a receiver which receives the position information about the object transmitted from the detection device,
    an acquirer which acquires position information about the object search device, and
    a display controller which shows a specific display to specify a position of the object on the display based on the position information about the object and the position information about the object search device.
    [0002] The object search system according to claim 1, wherein the display controller shows, as the specific display, a straight line indicating the position of the object such that the straight line extends to the position of the object.
    [0003] The object search system according to claim 1, wherein the display controller shows, as the specific display, a display indicating nearness to the object in the position of the object.
    [0004] The object search system according to claim 3, wherein the display controller increases the number of displays indicating the nearness to the object, or makes the display indicating the nearness become larger, as the nearness increases.
    [0005] The object search system according to claim 1, wherein the display controller shows, as the specific display, an arrow indicating a direction toward the object.
    [0006] The object search system according to claim 1, whereinthe detection device includes a camera which captures an image of the object,the object search device receives the image of the object, andthe display controller shows the image on the display.
    [0007] An object search device to be worn by a user, and including a display which shows an augmented reality space on a real space, comprising:
    a receiver which acquires position information about an object to be searched for;
    an acquirer which acquires position information about the object search device; and
    a display controller which shows a specific display to specify a position of the object on the display based on the position information about the object and the position information about the object search device.
    [0008] The object search device according to claim 7, wherein the display controller shows, as the specific display, a straight line indicating the position of the object such that the straight line extends to the position of the object.
    [0009] The object search device according to claim 7, wherein the display controller shows, as the specific display, a display indicating nearness to the object in the position of the object.
    [0010] The object search device according to claim 9, wherein the display controller increases the number of displays indicating the nearness to the object, or makes the display indicating the nearness become larger, as the nearness increases.
    [0011] The object search device according to claim 7, wherein the display controller shows, as the specific display, an arrow indicating a direction toward the object.
    [0012] An object search method comprising:
    an object position acquiring step of acquiring position information about an object;
    a display position acquiring step of acquiring position information about an object search device to be worn by a user, and including a display which shows an augmented reality space on a real space; and
    a specific display step of showing, as a specific display reflecting a relationship between the position information about the object and the position information about the object search device, the position information about the object in the augmented reality space based on the position information about the object and the position information about the object search device.
    类似技术:
    公开号 | 公开日 | 专利标题
    CN105842683B|2018-06-19|A kind of unmanned plane integrated campaign system and method
    JP2021012709A|2021-02-04|Navigating vehicle based on detected barrier
    US9977434B2|2018-05-22|Automatic tracking mode for controlling an unmanned aerial vehicle
    US9783320B2|2017-10-10|Airplane collision avoidance
    CN106249750B|2021-01-26|Method and apparatus for autonomously performing decisions to accomplish a task on an unmanned aerial vehicle
    AU2011352408B2|2015-11-05|Tracking moving objects using a camera network
    US8742952B1|2014-06-03|Traffic awareness systems and methods
    EP2495970B1|2015-06-24|Display image switching device and display method
    US10322804B2|2019-06-18|Device that controls flight altitude of unmanned aerial vehicle
    EP2856455B1|2021-01-27|Systems and methods for enhanced awareness of obstacle proximity during taxi operations
    EP1906151B1|2016-12-14|Imaging and display system to aid helicopter landings in brownout conditions
    CN105486175B|2017-05-10|Low-altitude security protection system and method based on large-power continuous laser
    EP3021078B1|2018-09-26|Geodetic surveying system with virtual camera
    US8195353B2|2012-06-05|Guide robot device and guide system
    EP2167920B1|2013-09-18|Aircraft landing assistance
    US4293857A|1981-10-06|Collision avoidance warning system
    US9688518B2|2017-06-27|Three dimensional rendering of job site
    KR101533905B1|2015-07-03|A surveillance system and a method for detecting a foreign object, debris, or damage in an airfield
    CN106839882B|2018-12-28|Special area invades unmanned plane early warning interceptor control system
    EP3245640A1|2017-11-22|An integrative security system and method
    CN104842358A|2015-08-19|Autonomous mobile multifunctional robot
    CN103941746A|2014-07-23|System and method for processing unmanned aerial vehicle polling image
    US20080195309A1|2008-08-14|System And Method For Aiding Pilot Preview, Rehearsal, Review, and Real-Time Visual Acquisition Of Flight Mission Progress
    US20130162838A1|2013-06-27|Transformation between Image and Map Coordinates
    US9472109B2|2016-10-18|Obstacle detection system providing context awareness
    同族专利:
    公开号 | 公开日
    WO2019065045A1|2019-04-04|
    SG11202001936SA|2020-04-29|
    JPWO2019065045A1|2020-04-23|
    JP6859446B2|2021-04-14|
    US20200193159A1|2020-06-18|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    法律状态:
    2019-04-06| STAA| Information on the status of an ep patent application or granted ep patent|Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
    2020-07-03| STAA| Information on the status of an ep patent application or granted ep patent|Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
    2020-07-03| PUAI| Public reference made under article 153(3) epc to a published international application that has entered the european phase|Free format text: ORIGINAL CODE: 0009012 |
    2020-08-05| 17P| Request for examination filed|Effective date: 20200302 |
    2020-08-05| AK| Designated contracting states|Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
    2020-08-05| AX| Request for extension of the european patent|Extension state: BA ME |
    2020-12-30| DAV| Request for validation of the european patent (deleted)|
    2020-12-30| DAX| Request for extension of the european patent (deleted)|
    2021-06-23| A4| Supplementary search report drawn up and despatched|Effective date: 20210520 |
    2021-06-23| RIC1| Information provided on ipc code assigned before grant|Ipc: G06K9/00 20060101AFI20210514BHEP Ipc: G06F3/048120130101ALI20210514BHEP |
    优先权:
    申请号 | 申请日 | 专利标题
    [返回顶部]